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ABSTRACT 

This paper responds to the demand generated among 
competency- based program participants and administrators for 
definitions of "cdmpetence»» and new assessment techniques and 
instruments for its measurement. Discussion centers on three aspects 
of assessment: (1) the identification and definition of competencies 
relevant to life and work, outside of academia; (2) instrumentation, 
techniques, and processes oif evaluation that provide reliable and 
valid measures of these competencies; and (3) standardization and/or 
establishment of levels of performance necessary and sufficient for 
awarding credentials. Technical problems in these three 'areas are 
identified, and directions for future research and development are 
suggested. Discussion strongly advocates an emphasis for competency 
assessment and identification that centers on adult development and 
learning outcomes with special attention to the' interactive nature of 
psychological variables and how skills and abilities are integrated > 
(as. is required in life outside of academia) . A "psychology of 
competence" is predicted. (MM) 
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Corollary 1: ''New'' competencies must be identified and. 



Corollary 2: 



Corollary 3: 



Corollary 4: 



operationally defined. 

New Competencies should have general significance 
to a wide variety of career and life outcomes. 

(a) The Problem of Reductionism 

(b) The Problem of Action-Orientations 

New competency definitions and measures developed 
for their assessment should be easy for faculty 
and students to comprehend and view as meaningful 
and useful. 

Competencies should be empirically linked to 
external, realities. 



B. Instrumentation, Techniques and Processes 



Corollary 5: 



We need to discover new ways of measuri-ng abil- 
ities (competencies). 

(a) New Approaches 

(b) Psychomotor Skills 

(c) Cognitive Abilities 

(d) Interpersonal Abilities 
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Standardization or Establishment of Levels of Performance 

Corollary 6: Standards of performance for awarding credentials 
should acknowledge levels of performance . required 
for entry into roles outside the academic setting 

, (a) The Problem of Maximum Levels 

(b) The Problem of Interactions 

Corollary. 7: New attempts to. define and assess learning outcomes 
should not be guided solely by attempts to make 
them functionally equivalent substitutes for trad- 
itionally assessed school achievement. 
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INTRODUCTION 



^^^^^^^^^^^^.^^.^^^^^^^^^ ^.^^ point "of 'dep^tufe"^Tie aemarid beifig 
generated among competency-based programs for definitions of "competence" 
and new assessment techsniques and instruments for its measurement. 

Manjr competency-based practitioners believe that traditional approaches 
to student evaluation (e.g., multiple choice paper- and pencil-tests and 
written papers) are not suitable devices for assessing student progress in 
many spheres of activity of interest to them. They are, therefore, engaged 
in developing new techniques and are calling upon psychometricians to assist 
them. 

These educators might have been cp:iitent to simply define or redefine 
institutional and program* goals, charge the faculty with their adoption, 
and not concern themselves with testing. Instead, in many cases, a super- 
structure of student assessment is being introduced (in two- and four-year 
postsecondary programs) in whibh faculty evaluations of students at the in- 
structional-unit level play only a partial or minor role, if any role at all. 
Even where such a superstructure is not introduced, faculty are being re- 
quired to state explicitly th| criteria upon which student achievement will 
be judged, and some sort of monitoring -and assistance system is being in- 
troduced to assure the appropriateness (relative to the instructor's goals) 
of assessment instruments and criteria of evaluation employed. Why? 

The fear is that unless these steps are taken, the time, effort, 
and money spent in formulating or reformulating program goals will have 
been wasted — that things would not change much if they changed at all. 
This is unacceptable since most competency-based practitioners have 
energetic reform agendas. The reasoning behind their pessimism (about 
the likelihood of changing the form arid content of education without 

1 
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changing the torzx and coiiLeiu of ajisessraeac) is akin to the folloving: 

Faculty habits of instructing (content f ecus, " skill 
emphasis and pedagogical techniques employed) are not 
.Qhanged^ by simple requests or^^loqueat s.upp.licatlans.. 

Where program-level tests exiet which supercede the 
assessments individual instructors devise, there is 
a tendency to teach to such tests (or at least vorry 
about the consequences of not doing so). 

Given tests in courses or at other levels of a prog::ain, 
students tend to gear their studies to these tests. 

Therefore, if faculty are to change their habits oi 
instruction and students are to change the focus of 
their attention, then "tests" Students are required 
to take must emphasize dimensions of ability commen- 
surate with the avowed goals of the institution, pro- 
gram, or particular instructor.* 

What explains the- equally strong emphasis on goal clarification and 
operational specification? \Jhy the phenomenon of old and new Institutions 
busily engaged in formulating new goals or resurrecting and polishing up 
old ones? 



Many of the new competency-based institutions or programs in univer- 
sities are geared to special populations such as women, racial or ethnic 
groups, adults, or future occupants of particular occupational roles. 
Some of the, old institutions wish to attract these same special popula- 
tions and to attract a greater number of "traditional" students* Moreover, 



(1) 
(2) 

(3) 
(4) 



Of course, several bther explanations are possible Including following the 
leader : "That's what all the other competency-based programs do — don't ask 
why?" Or , "we don't trust the professors here with grading students. They 
want to give everybody an 'X', and we won't have it I The efficacy of 
our degree is at stake." But,— at—this-point—in—the histo,r*- of competency- 
based education the explanation outlined above seems most ^-.^lausible. 
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"evenXir^and ei:ief logics of'tJid " pas t* 'Elccalie^ ot'^Qo'treeu^ 
have produced a great zeal for educational reform at all levels of the system. 
Attention, therefore, is being given to what sort of education will be 
offered: (a) to attract new student populations, (b) to continue to attract 
-the-~tTadi-tioaal student population, and {e-)-'tGu.satisfy the reform agendas 
of program founders. This leads to a discussion of educational goals. 

As explained in the commentary accompanying this set of papers, some- 
where in :the course of planning for change, many institutions have adopted 

/ 

the name "competency-based." In the process of translating broad educational 
goals of the institution into concrete programs and curricula, "goals" have 
come to be Interpreted "competencies" or "performance objectives." When 
the focus becomes "competency, *'\^e following questions are raised: 



(1) What does it mean to competently or to 

be a competent X? 



(2) How do we determine whether students can 
competently or whether they are competent 



(3) What do we do as educators to promote these kinds of 
competence? 



As already discussed, the answer to the second question (and a partial 
answer to the third) has been: "We assess the students. We ask them to 
perform competently for us." 

\ 

Undertaking the development of assessment criteria, techniques and 
instruments and arriving at^ decisions about appropriate programs compels 



* ■ . 

Some competency-based educatioiial practitioners have recognized that many 
important "goals" of* education do not translate into "competency" objectives 
and that care must be taken not to equate the two; others have not. "This 
can have significant consequences for the curriculum and environment of 
the institution or program which emerges from educational plans. 
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acLericion back: to che first question and, of necessity (with respect to 
assessment), to the operational definition of ''competence" or "competencies. 

The paper which follows, by Paul, Pott Inger, emphasises a need for 
systematic empirical research in connection with competency identification , 
specification ^ and measurement . It identifies technical problems in 
these areas and promising directions for future research and development. 
The paper strongly advocates particular approaches to competency identifi- 
cation, specification, and measurement requiring systematic empirical 
investigation and verification; and, in doing so, it presents an indirect 
critique of the current state of the art in these areas. It will be 
difficult for those who lack the contrast class of present realities — a 
working knowledge of what is and is not..being offered to, assist educators 
in identifying, specifying, and measuring competence—to fully appreciate 
the recommendations in this paper. 

.Sheila M. Huff 



The latter term, "competencies," is used variously to mean "component ele- 
ments of coiLpetence" and "areas of competence.", There is considerablfc 
semantic confusion in the emerging technical langi^^ge of competency-based 
education and a good deal ojf unnecessary linguistic" invention that makes 
many familiar ideas sound foreign and, to some, offensive. 



PROBLEMS, ISSUES, AND GUIDES FOR RE SEARCH , 

The problems s.teimning from the identification and definition of .coia-^ 
petencies and subsequent problems of their measurement are at the core of 
many difficulties reflected in the current state of the /art of competency 
assessment. Specifically, there are three aspects of assessment upon which 
this paper will focus: 

/ . ■ 

• The identification and definition of competencies 
relevant to life and work outside of a^cadiemia; 

• Instrumentation, techniques, and processes of evaluation 
that provide reliable and valid measures of these compe- 
tencies; J 

. Standardization, and/or establishment of levels of perform- 
ance necessary and sufficient for awarding credentials.* 

The reader should bear in mind that these major assessment issues are 
complex and interdependent. Therefore, the following sequence and organiza- 
tion of major topics and their corollaries are somewhat arbitrary, at best. 
However, some broad parameters follow which hopefully will giiide researchers. 



* 

There are other considerations that should be taken into account in the 
development of new competency definitions a.nd measures which, for most 
postsecondary institutions, have great enough importance to be kept keenly 
in mind by competency specialists. These include: . (1) cost/effectiveness, 
(2) didactic or pedagogical values of measures, (3) ease of interpretation, 
(4) efficiency of use, (5) psychological impact of taking these tests. 
These issues will not be ma^jor topics of concern here; suffice it to say 
that new measures should be economical with respect to both faculty time 
and institutional financial resources, since measurement techniques, in- 
struments or procedures which require large amounts of time and financial 
resources will have limited appeai^ and practical application. Sim5.1arly, 
measures which lack face validity carry a threatening mystique to those 
who do not understand them, especially if they lack clarity in their 
correspondence to specific goals. 
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i Geati t"i cation a nc^ Definition of CoiniieCeaclcS 

Corollary jL: -%ew^^ competencies Liust be identified 

- * 
and opeirationally defined. 

. * \ ' 

There are many outcomes of learning not functionally equivalent to : 
those upon which credentials are normally awarded and which often have greater 
legitimacy with respect to national educational priorities f'>r all students 
than traditionally rewarded scholastic and athletic ability. AlthoiLigh widely 
accepted (and validated as important in occupational atiid life success by a 
growing body of empirical evidence), there has been a failure to identify, 
operationalize, measure, and award credit for many of these learning ouC^Omes. 
Some of these include such intuitively accepted ingredients of success as the 
ability to relate to others effectively, to empathize with others, to enii^age 
in moral reasoning, to accept responsibilxty,^ to ^^rsevere, and to analyze, 
synthesize and strategize in problem resolution. 

The development of these abilities and their concomitant behavioral 
dispositions is usually not a subject of academic planning, although they do 
continue to develop (or cease developing) in the environment of colleges and 
universities and are of concern to many instructors and administrators. As 

'k'k 

Keeton (1974) points out: 

...although achievement of such skills is often an avowed aim 
of the most traditional liberal arts colleges it is rarely 
certified by their credentials, and is of ten disavowed by 
instructors as something which cannot properly be expected 
as -an outcome for which they can rightly be held accountable,/ 



To operationally define is to say what behaviors will be taken as evidence 
that a student has a given ability. The meaning of the ability, fpr opera- 
tional purposes, becomes the criteria by which the ability is verified. 
For example, the meaning of "ability to communicate effectively," for 
operational purposes becomes: "the student can write papers meeting the 
following criteria: ." and "the student can deliver speeches meeting the 
following criteria: ... ," etc. c 

'Full citations, appear in the "References," pp. 26 to 27. 
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Many conipo.teacy-basod progr.iins enter .these and other *'naglccted" abllitleo 
on their list of desired learning outcomes for students. These programs 
"wish to promote the development of these abilities in the curriculum and 
institutional eavironiRent and to assess and credit student progress in 
developing these abilities. If this were accomplished, it would provide 
nontraditional students with an opportunity to pursue credentials in areas 
relevant to their lives which traditionally have not been a basis for, the 
conferral of degrees. ° 



(a) The Problem of Reductionisia 

Competencies cannot be meaningf ullly defined by seemingly endless re- 
ductions of specific skills, tasks and actions which, 4n the end, fall 
short of real world requirements for effective performance. In fact, the 
more essential characteristics for success will often turn out to be broad 
or generalized abilities. or characteristics which are sometimes more easily 
operationally defined and measured than an array of specific "subskills" 
which do nbt add up to a general competence. 

In many competency-basad education (CBE) programs, attempts are made 
to reduce competencies to a series of discrete and hopefully quantifiable 
action steps. This reductionism follows from the need to clearly communi- 
cate as well as to t^uantify and measure outcomes. However, from the 
students' point e? view, a myriad of overly reductive definitions is awesome 
and the definitions* t!^s1^ms elves, often lack intuitive meaning; i.e., the 
"overkill" of subcompetencies lacks the same sense of meaning and relevance 
to students' lives as the traditional learning agendas from which many have 
fled. To the student who asks, "What do I have to be able to do to be 
competent; what do I have to demonstrate in order to be credentialed; and 
what do these exhibited abilities have to do with the real world?,^^ the 



Corollary 2: New competotncies should have general Significance 
to a wide variety of career and life outcomes . 
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curreaL' state of the art in defining competence sometimes affords a re- 
grettably inadequate answer. 

From the instructor's point of view, too, such definitions seem to 
have forfeited what was important in the institution's educational goals. 
Many instructors justifiably resist demands to teach "competencies" that 
are^too specific, narrow, or intuitively and empirically of xninor importance 
in life. — ' 

Knott (1975) has addressed this issue: 

....Competence as a goal of liberal education refers to an 
overall characterization of the individual rather than to the 
possession of discrete coiapetencies* 

Mastery of a set of specific competencies does not necessarily 
produce a liberally educated' p:^jrson» Specific competencies 
must be synthesized or integrated into an effective vhole 
which is more than the sum of the parts. The. concern of 
liberal ation refers to the competence -of the person 
rather than the collection- of p'bssibly unrelated competencies. 
...the emphasis in assessment of students in a competency based' 
liberal ed ucation is holistic rather than f ragiiuented ^ If 
assessment criteria and procedures do not reflect this emphases 
and focus instead on specific unintegrated .competencies, the 
primary goals of liberal education may be, at best, casually 
overlooked or, at worst, systematically ignored. 

(For further elaboration of this point, see section on the 
problem of interactions, p. 20.) 

c 

(b) The Problem of Action-Qfientations . 

Although Knott expresses concern that competency -based assessment not 
subvert the basic intent of "liberal education," his point is well taken 
for more career-oriented programs where the specification of narrowly 
def ined-'behaviors are, for the most part, equally inadequate in preparing 



/8 , 
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pttuplo ioL' work. Educators in career-oriented schools have often turned 
to the methodology and assessment techniques developed in industrial 
psycholo^";y, but most of these techniques exclude broad dimensions of ability 
vital to the performance of most occupational roles. 

According to McClelland (1974), current approaches to educational pre-r . 
partition for jobs, based :on the type of job analysis most widely' practiced, 
have serious deficiencies at the theoretical and practical levels which 
have not been widely recognized (e.g.. Fine and Wiley's [1971] analysis for 
the Directory of Occupational Titles ). The initial assumption of this 
approach to so called "job fiinction analysis'' is so severely limiting that 
it simply might as well be labeled "incorrect." Example: 

<r^ "A j'ob is made up of a series of tasks. 

0 "A task is an action or action sequence*" - 

The action orientation of this approach has blinded practitioners to - 
competencies which are absolutely essential to many jobs. The job function 
analysis approach is based on motor skills' analysis and has utility in their 
identification; but it is too narrow an approach to be used as a method, for 
determining significant dimensions of job competence • 

This approach to job\ analysis results in taxonomies of hundreds, some- 
times thousands, of motor ^^kills connected with particular kinds of jobs. 



"k \ 

While this paper's f ocus^ is primarily upon job preparation, it is so only 
for convenience of ,anai3''sis and discussion. There is undoubtedly consid- 
erable overlap between abilities required for competent performance in jobs 
and abilities required for other of lifers activities. Simply consider 
typical liberal arts'x goals of art appreciation, cultural awareness, and 
social sensitivity and you will find jobs V7here these qualities are«=^-Ji^hly 
valued . 
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These taxonomies art* frequently used in developing the curricula of occupational 
education prograins. For other reasons besides the neglect of niany significant 
areas of job coapetence, such taxonomies are not suitable guides for educators- 
For example, there is a considerable risk of forgetting that many — perhaps in 
some cases, most — of these skills can be picked up on the job in a abort period 
of time and are therefore not worthy of attention in educational progracos- 
Curricula would be more relevant if they addressed a variety of nrre general 
abilities essential to competence. 

Corollary 3: New competency definitions and measures developed for 

their assessment should be easy for faculty and students . 
to comprehend and view as meaningful and useful . 

New competency definitions should be readily recognizable as important, 
and the related assessment techniques or instruments should be easy for faculty 
antl students to understand. There is a need to guarc' against competency defirii- 
tions and measures that are so complex, trivial, or esoteric that students and 
faculty cannot, in the first instance, understand them and, in the second, 
accept them as meaningful and usfiful. In other words, educational goals 
should not be rendered unintelligible; and assessment procedures and instruments 
should not mystify the process of evaluation of student progress. Avoiding 
these pitfalls may require the participation of "non-experts" in the field- 
testing and review of proposed definitions and assessment measures. 

Corollary 4: Competencies should be empirically linked to 
external realities . 

Many ed ucators take it as obvious that such things as the ability to 
master new bodies of knowledge quickly and effectively, to analyze and solve 
problems, to develop new skills efficiently, to utilize knowledge, etc. (and 
the capability of integrating these abilities) are critical if individuals 
are to take advantage of life's opportunities and sum^unt its difficulties. 
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Unfortunately, even though students might assent to the importance of 
.these abilities, in their eagerness to develop ''marketable skills," they may 
not see such abilities as saleable. And although these kinds of ability may 
indeed make the difference between those who do and those who do not advance 
in a career and may, indeed, be highly valued by employers, a growing number 
of students do not believe that to be the case. Students, especially in 
.tizies of high unemployment, often have their eye on getting a job; see that 
they n-eed. a credential; think employers prefer technical credentials; and 
look to the quickest route to a credential through a program with very circun^ 
scribed knowledge and skill objectives. School administrators often respond 
in kind, leaving many faculty members out on a limb — unable to demonstrate, 
and even sometimes to articulate that much of what they do promotes general 
abilities that are, perhaps, the roost "marketable of skills" and most valuable 
of abilities in terms. of future growth and flexibility. 

What is needed are measures of these" general abilities and their inter- 
actions which show they are related to important life outcomes. Only when we 
know what makes the difference between adequate and inadequate performance, 
based on en^irical analyses of jobs and other life activities, will we be 
able to develop or improve these measures, clarify new competencies, and 
value credentials based upon them. 

The lack of empirical data about what constitutes competence is well 
illustrated by the current selection and evaluation procedures of profes- 
sional schools. While the debate about the effectiveness of aptitude tests 
for selecting and evaluating students who will perform well in law, medical, 
and business schools goes on between academicians, psychometricians anci the 
courts (e.g., the DeFunis case) , virtually none of the participants have 
addressed tl nselves to the question of how well these tests predict success- 
ful o^ccupational performance or competence. There is an abundance of empiricrvl 
evidence which shows that doing well on these tests or in subsequent schooling 
does not predict success on the job. 



11 



14 



Perhaps those who support the Medical College Admission Tost, the Law . 
School Admission Tes;t, and the Admission Test for Graduate Study in Business 
as. significant prtidictors of academic performance miss the point. We need 
to discover the skills, abilities and other characteristics critical to 
competent, post^academic professional performance and develop reliable and 
valid measures of these, variables to supplant or supplement existing selec- 
tion, assessment and evaluation tools. Until we do, we will continve limiting 
access to professions to only those who can detconstrate high achievement on 
a narrowly proscribed set of cognitive measures deemed essential for academic 
performance alone; while most often, it is abilities like those listed above 
and characteristics such as motivation, perseverance, dedication and integrity 
that separate the more competent professionals from the rest of their col- 
leagues. 

Instrumentation, Techniques and. Processes 

Corollary 5: We need to discover new ways of measuring abilities 
(competencies) . 

They must be new, not just new names for traditional procedures. 
Achievement on traditional paper-and-pencil> objective tests correlates 
highly with performance on all similar types of academic achievement tests; 
but if the desire is to break out of this closed circuit, there is a need 
for radically new type© of "tests'" — tests of learning, critical thinking, 
problem solving and other newly defined competencies which correlate with 
competent performance in jobs and other nonacadenic situations. 

^ (a) New Approaches 

Within the competency-based movement, many innovative approaches to 
assessment are being developed, miany of-^hich" borrow from techniques and 
procedures developed by industrial^'psychologists"- For example: 

12 



- portfolios ' --^ 

- journals 

- juries 

. - committees 

- life histories 

- self-assessments 

- supervisor* peer and/or client ratings . . 

- in-basket tests 

- work sample tests 

- games 

- simulations 

- projects .« 

- contests 

-- rehearsed performances ^2 

These attempts to break away, from the limited traditional measures of verbal 
ability and scholastic aptitude and achievement have sometimes resulted in 
elaborate, time-consuming, costly and cutnbersoiae techniques and procedures; 
and Tnost of these assessment techniques are quite subjective. They are not 
amenable to standardization for comparability among individuals and institu- 
tions. ' o 

The major effort underway by ETS (Cooperative Assessment of Experiential 
Learning — GAEL) to develop new procedures for measuring performance related 
to a varxvty of competencies is one attempt to break away from traditional 
measures which are method bound, limited in scope, and of no demonstrable 
relationship to competent performances outside of academia. CAEL's emphasis 
on performance measures of learning outcomes is, in itself, a sound approach. 
However, these new measures suffer from some of the same shortcomings of 
traditional tests. That is, (1) the techniques tend to be highly subjective 
and open to broad interpretation; (2) they dp not easily lend themselves to 
standardization across institutions or even among:.indivi duals who use them; 
(3) there is as yet little or no empirical evidence that the performances 
being measured are any more related to success outside of academic than per- 
formances measured by traditional means. Moreover^ these new procedures and 
techniques do not appear to lend themselves to rigorous empirical reality 
testing. 
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Until a hose, of 'measures are developed that are reliable, valid, standar 
ized and rigorously demonstrated to be directly linked to significant life 
activities, evaluations and credentials based upon these new performance 
measures will have little meaning beyond particular institutional settings 
and will, therefore, not gain wide acceptance. • ^ 

There are few examples of empirically derived competency measures (which 
have also been adequately linked to successful performance in work and life), 
but brief comments related to some that do exist follow. 

(b) Psychomotor Skills 

The most advanced state of the art is in the area of ..psychomotor skills . 
The Human Resources Research Organization (HUMRRO) > for exan^le, has accoin- 
plished a great deal with its technology and has collected a vast amount of 
interesting data. However, as previously; pointed out, psychomotor measures 
are of limited utility with respect to CBE. / 

(c) Cognitive Abilities 

Cognition has received much attention by researchers and educators. 
Traditional tests of critical thinking, analysis, and problem solving have 
sometimes been valid and consistent with stated learning goals, but these 
goals have been mostly determined on a yriori grounds and thus limited with 
respect to their demonstrated relevance beyond academla. For exainple^, not 
all educators realize (or act as ^though they realize) that the ability to 
store knowledge (which most tests measure) is less critical than the ability 
to utilize It. Similarly, the ability to recall information may not be so 
critical for. many life functions as, say, the ability to acquire new informa- 
tion quickly". 

Some of Klentp's (1974) thoughts .on the matter are illustrative: 



In our daily lives we are constvintly called upon to process 
various kinds of information, to analyze its compon^-nts , to 
associate this new information with that which we have stored 
away in our mencory, to partial ouC- the crucial information from 
the triyial, and to integrate r' .uS information into our cog- 
nitive structure. In this wa^ we constantly use information 
from many sources to solve problems, and in the process we learn 
new things about our world and ourselves. It would serve us 
well to ask the extent to which a multiple choice or true-false 
test, has any bearing on what people do in real life and on the 
competencies that they possess. In truth, people are aliiiost 
never asked to recognize a correct answer among a list of three 
or four alternatives^ Rather than being reactive to such a well- 
defined situation, people must be pro-active in situations which 
provide only partial information. 

The one thing most traditional testing mtithods have in common, 
.regardless of what they purport to assess, is this: they measure 
only one's ability to retrieve info rmation...after it has been 
stored. And many such methods fail even in this; a multiple- 
choice test, for example, measures the ability to recognize 
rather than recall. ESsay tests are very subjectively scored, 
even when there is only one "correct" answer or line of reasoning 
as is often the case. Storage and retrieval of information are 
not the important issues for i competency-based program of study — 
nor should they be for traditional programs. Indeed,^ Ebbinghaus 
demonstrated many years ago that seventy percent of what is 
learned in the classroom is forgotten within^ one year.' Rather, 
the issue is a more substantive one: how is the knowledged gained 
in course work used to come to grips with practical problems. of 
living. 

^ Impllcit^n this, according 'to Klemp, are three related issues of par- 
ticular importance Zz-'liow able are people in processing new information for 
problem solving; how able are, tKey in integrating this information to form 
new, solutions; and how effective are they in implementing these solutions. 
Klemp and his colleagues are developing innovative measvires of these critical 
cognitive skills which should h'ave' wide applicability t raditional as well 
as CBE prpgrams. ^ 

. Others have attempted to define critical dimensions of cognitive com- 
petence. Knott (1975), for example, proposes .three clusters, ofr-abilities as 
desired outcomes of an effective liberal education: . (i) the ability to formu- 
late and examine purposes; (2) the abiljity to. design and act upon means of 



executing those purposes; and (3) che ability to assess consequences for them- 
selves and others of designed action on selected or formulated purposes. 

Aubrey Forrest (paper undated) of Minnesota Metropolitan State College 
has identified three broad dimensions of competence: knowing, applying and 
evaluating. 

Although these and other domains of competence appear among CBE objec- 
tives, there are no existing measures of them that have construct and/or 
empirical validity, reliability, or cri.terdx^n levels linked to performaaces 
which represent occupational or other life competencies. However, the research 
underway by Klemp in developing such measures and in relating them to perform- 
. ance outside of the academic world should provide replicable mbdels for 
developing new cognitive competency definitions ■ and measures.* 

(d) Interpersonal Abilities , 

Very little attention has been given to the measurement of interpersonal- 
. dimensions of competence. Tlie following work is among the most promising 
done to date: 

(1) In a study of U.Sc Information Officers, McClelland (1972) found 
that more successful officers scored higher- on -a measure of non- 
verbal sensitivity derived from the PONS test by Rosenthal et |1 . 
(1974). These individuals were better at identifying the emotxons 
expressed in content- filtered speech;, and this ability was found., 
to correlate with au understanding of how audiences would react., 
. - to various media presentations. The latter kind of understanding^ 
' -labelled "empathy "--was found to be critical for the competent 
performance of this job; and part of the PONS test apparently 
measures it well. 



Another example of research worthy of.analysis is that ^'eing done under 
B,-an Whitla of Harvard University. Harvard's Project Valun Added is 
u-ilizing some interesting new competency-based learning outcome ftieasures 
to evaluate the effects that three college programs have on students 
according to freshmen/senior^ comparisons on these new measures. 
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(2) In an aaalysis of human service v/orkers and police officers, 
McClelland (1971, 1974) found that observational abilities 
are of major importance — particularly in social situations. 
He prepared filmed incidents in which people were asked to 
identify and correctly report what occurred and make judgments 
about what they would do in the situation. Learning to 
obsei-^re and interpret human behavior is an important outcome 
of many educational experiences, but it is rarely measured 
objectively or credited. 

(3) Stewart (1975) has developed a measure of proactive style . 
She demonstrated that, for women, a proactive disposition 
is increased by some colleges. However, it was discoverea 
that increases in the ability to be proactive may create a 
fear of success in woinen who, as a consequence, becon,. less 
able to pursue their .chosen areers. Stewart is the only 
work in this area that has actually linked fear of success 

to act^oal career patterns of women. Jacquelin. Flemming (1974) 
of Radclif fe College has been leading the research on fear- 
of^success as it applies to Blacks. 

The continuing research of Stewart, Flemming and others should 
prove fruitful ground for identifying, operationalizing and 
measuring a variety of variables inportant to successful life 
activities and amenablecto development or change through 
education. - 

(4> The problem of adequate selection of medical school a^jpllcants 
is similar to that of evaluating student learning, especially 
when one's goal is to select applicants who will not only do 
well in medical school, but who will also make'good practi- 
: tioners. It has long been s.uspected that traditional selec- 
tion procedures have been inadequate; measures such aff under- 
graduate grades, MCAT scores, background and interview ratings' 
of applicants, and other such measures bear little relation 
. to whether an applicant will eventually be a competent doctor 
from either his own standpoint or the standpoint of the patients 
he serves! Recognizing these problems," the Association of 
American Medical Colleges (AAMC) is sponsoring research and 
development of competency measures that relate to socio- 
eiDOtional, motivational, moral, and behavioral outcomes as 
well as more traditional cognitive variables in learning. 

The methodology in this new research will move beyond the 
often used industrJ.al psychology model that typically pre- 
scribes a list of selected areas of knowledge, skill, ability, 
or personal qualities derived- exclusively from audits, sur- 
veys, questionnaires, or task analyses. For example, systerat- 
a tic analyses of critical incidents of success and failure 
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will be performed to reveal patterns of beha\dor, skills, 
and especially other'^characteiristics (e.g., non-cognitive, 
socio-emotional, personal qualities) that workers and 
experts themselves are unable to report as important to 
success. Going beyond traditional reliance on expert 
testimony or self-reports is necessary becaiise most people 
have an incorrect or very limited understanding of (and 
ability to articulate) the elements of their work that lead 
to successful performance. 

AAMC's research might well provide researchers with . another 
model for identifying and measuring new competencies that 
are empirically linked to successful postacadeinic perform- 



ance. 



C. Standardization or Establishmaht of 
Levels of Perf ormaiTd'ei' -. 

Corollary 6: Standards of performance for awarding credentials should 
acknowledge levels' of performance required for entry into 
roles out-::>ide the -academic setting . 

The determination of criteria or standards of competence is one of the 
most difficult problems to be addressed. In every case, whether standards of 
competence ul new, or more traditional outcomes are determined, appropriate 
levels should be established by empirical evidence sufficient to ensure that 
they will not be viewed as arbitrary, Many educators have been satisfied 
with a priori judgments of what skills and levels of performance are adequate. 
It is startling to realize how much we accept the face validity of creden- • 
tials and how little we really know about the correspondence between abilities 
and levels, of performance these credentials' represent and what in fact is 
needed for adequate performance in life's tasks. We have no sound benchmarks 
for evaluating the standards and offerings of postsecondary instituclons* 

(a) The Problem of Maximum Levels 

• ' Credentials are often restricted to those whose scholastic performance - 
and/or test scores are higher than minimal levels required for work or other 
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social roles. Such occurrences discriminate junfairly against those who are 
competent to work, for example, but who/are selected out of occupational 
opportunities by those who believe in the simple equation: higher academic 
achieveraent means better work or life performance. The tacit assumptionP that 
superior abilities in all measured characteristics are necessary or even 
desirable for performance is highly questionable.* 

While it makes sense to require minimal levels of proficiency for many 
competencies, ability levels over and above necessary cut-off points do not 
always correlate with overall performance. . 

For example, in a job analysis, McClelland (1974) found that a minimal 
level of organizational or clerical competency was necessary for human service 
workers in the Massachusetts Civil Service system, but high. scores on these 
measures were negatively correlated with superior job performance . Selecting 
people 'by rank according to score not only discriminated against those whose 
scores were adequate (sufficient) though "uncompetitive," but the process 
failed to select the better job performers as well. This finding and others 
suggest that going beyond sufficient levels of competency in awarding creden- 
tials can be very dysfunction;:l for society — not only in ter.ms of equity, but 
in terms of meritocracy as well. 

In many job situations, where cognitive and other competency measures 
are used to select job applicants, even if job relevance of. the characteristics 
being tested for can be demonstrated (e.g., "verbal ability" in human service 
workers), level' of sufficiency for competent job performance is rarely evalu- 
ated or known. 



A simple motor skill example will demonstrate this point. We know that an 
automobile driver must grip the steering wheel with enough force to'^maintain 
control of the car. But beyond a certain level of pressure, added strength 
in holding the wheel does not increase overall driving competency. And 
this is just one of some 3,400 discrete behaviors identified by researchers 
as making up the task of "driving." 

.A recent study at Harvard revealed that the past. SAT scores of faculty members 
were negatively correlated with more successful teachers. (Whitla, personal* 
communication. ) 
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We need loore empirical research to establish minimal levels of competence / 
required for quality performance based on hov workers in the field perform on 
various competency measures. 

(b) The Problem of Interactions 

• v.;*' 

Researchers have long recognized ,that the interaction effects of variables,,^ — 
are quite often more significant and meaningful than individual variables taken 
alone. It was stressed earlier,, in quotations from Knott and Klemp, that com- 
petence is not a simple summation of discretely defined skills and abilities. 
This is readily seen in the example of driving ability. Although one can identify 
many skills necessary for safe and effective driving — including attitudes, cog^. 
nitive skills, and emotional factors, as well as perceptual . and motor skills- 
it is intuitively obvious that a simple summation of measurement scores on 
these discrete task performances; would not add up to equivalent driving skills. 
An individual who' is overly competent at some driving skills but woefully 
inadequate in others would be a poorer driver than someone whose skills were 
all 1 efficient, though- their summed skill scores would be identical! 

Measures typically used to assess job task performance and performance 
relating to the mastery of units in a curriculum typically have kittle bearing 
on how' subunits interact. For any given job, life task, or individual per- 
formance, component skills in one area can conpensate. for deficiencies in 
others creating a variety of combinations .of individual performance levels 
which could theoretically "add up to" equivalent overall performance. Thus, 
minimal levels of performance on individual variable3--jCwhi^^^ compromise over- 
all competence) may have little meaning by themselves . % Their interactions 
with respect to outcomes may have far greater significance. 

We are most familiar with this problem in cognitive areas of education. 
We are often taught language us.e, verbal reasoning, spatial relationship, 
reading coii5)rehensiori, abstract reasoning, and syllogistic analysis (measured 
by Miller Analogies) as discrete units of curricula. Assessment of integrated 

-20- - 

^ / ■■, ■ ■ 



or gemeral skills such as problem solving ofcen do not: take into account: th-^ 
interactive nature of skills iti these subcomponent areas. Cognitive measures 
are used almost exclusively in assessment as if the qualities they measure 
did not interact^ i/e. , they are tested separately. 

The importance of interactions, while intuitively obvious in the motor 

skills area, have not been carefully attended to in cognitive and social/emo- 

tional areas of assessment. Yet, once individuals have gone through a series 
of academic life^ experiences that enhance their competence in dealing with 
school, work,', and other lif^e experiences, the appropriate assessment task 
becomes that of measuring such integrated and generalized learning outcomes 
as the ability to cope with new problems, to find appropriate solutions, and 
to take the correct actions, 

Measures which reflect the interdependent nature of cognitive skills 
essential for satisfactory functioning outside of academla have only begun, 
to be developed,* For example, Klemp's (1974) General Integrative Model of 
Assessment,** incorporating a variety of independent techniques, is an approach 
to summative evaluation of an individual's ability to solve a problem which. ^ 
has as many elements. and conplexities of real life situations as possible. 
Such an assessment of individuals has the potential of coining closer to tap- 
ping real life competence than can any single test alone. 



A recent example in^ the noncpgnitive area by McClelland and Bumham reports 
the iDq)ortance of the.^ interaction between levels of motivation and ego- 
maturity for managerial competence, ( Harvard Business Review ; in ptess,) 

This general model requires an individual to demonstrate hisyher ability 
to integrate the following abilities: (a) ^ to observe; (b) to extract . 
relevant information; (c) to analyze and integrate this in format ion; (d) to 
ask appropriate questions; (3) to. process new information in response to 
such questions; (f) to utilize tTiis information and one's knowledge In 
making sound and logical recommendations; (g) to develop main and contingency 
plans.; (h)^to'set meaningful goals; and (i) to feed back this new information 
into the process for better problem analysis and solutions. 
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The implication for CBE is that one cannot as suae that abilities or skills 
discretely learned will be integrated in work and life functions and conse- 
quently that establishment of minimal levels of performance on isolated si ills 
or "sub- competencies" have much meaning in themselves. Therefore, competency 
research, new assessment procedures, and test instruments must focus more on 
the i;interdependence of skills. Basic research as well as empirical analysis 
of these interactions in various Irfe functions is desperately needed. 

Corollary 7: New attempts to define and assess learning outcomes should 
not be guided solely by attempts to make them functionally 
equivalent substitutes for traditionally assessed school 

achievement. ; 

■' ■ ■ ' " 1 

This statement should go without saying, given what has already been said 
about the inadequacies of traditional approaches to achievement assessment. 
However,' the temptation to restrict the development of new measurement instriip 
ments, techniques, and procedures in order to achieve comparability with \ 
those that have .gone before has great political appeal for making such innova-\ 
tions palatable to traditionalists. If institutional' and credential reforms \ 
are to succeed, we need to move beyond the recognized limitations of tradi- \ 
tional systems. 
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Ill 
CONCLUSIONS 

A. Implicatio ns for Research 

~~ " ^ ■•■ / 

The Implications for research are numerous. The tt4ed may be foir^rio'T^ess 
than a new psychology of coiqpetence—somethlng on the^order -pf"Blo^ and 
Krathwohl's taxonomies of cognitive and affective' dimensions of learning. 
But the emphasis must be on adult development and learning outcomes with 
special attention to the interactive nature of psychological variables and 
how skills and abilities are Integrated (as life outside of academia reqv^r 
It's a tall order, but a psychology of competence is beginning to emerge. 

Research by HUMRRD,* ETS, Knott, MqClelland, Stewart and others has 
already been mentioned as illustrative. Other, approaches—including Norviire^ 
Northcutt's survey of life skills (University of Te^as, ongoing), work in 
Oregon (State Department of Education, ongoing), and in Syracuse by Dr. Ruth 
Nickse (Regional Learning Seryice/SURC, ongoing) — represent attempts to 
define^ performance-based learning outcomes for high school' level competencies 
based on analysis of life skills. . The Center for Applied Performance Testing: 
represents a recent atten^t to build a "catalogue" of performance-based 
measures. While many of these latter attempts to define -and measure learning 
outcomes according to what people can do, may be restricted in scope, lack 
rigor or poorly correlate with job and life requirements, these attempts 
represent useful beginnings. / 

The current state of the art in assessment, howeVer, calls for more 

i' • 

conceptual rigor, more systematic and comprehensive strategies ; for identifying 
operationallzlng, and developing measures for new competencies, and more - 
empirical verification of their utility for a varletjr/of life Jfjinctions. ' 



Until we have a more coiq)rehensive base of empirically identified, 
clearly defined', and adequately measured competencies, educators will con- 
tinue to use an existing array of questionable measures based on narrow 
cognicive outcomes or on a priori value-^laden judgments. What is required 
is a reasonably sophisticated technology capable of uncovering knowledge, 
skills, abilities, and other'^characteristics which are necessary and suf- 
ficient (as well as ^thorough and efficient'*) for con^etent performance. 

B. Implications for Change 

The heavy emphasis on en^irical analysis and verification in this paper 
should not be taken as a denigration of educators having strong convictions 
about what constitutes quality education but wl|io are unablfB to empirically 
validate these convictions. Nor is there any intention of belittling those 
who assess student competence on a very subjective basis (that is, "I know 
competence when I see it"). Clearly, there are many capable individuals in 
education whose judgments of others are valid and whose evaluation efforts 
serve students, their institutions, and society well. The plea for more 
empirical research suggested or implied by the issues discussed in this 
paper stems from the belief that such research is critical to the development 
of quality QBE programs that attempt large-scale change in the way we reach, 
teach, assess and credential students to assure them .more productive and 
satisfying lives, bforeover, the outcomes of assessment research might well 
be the "prime' mover" in accomplishing the changes desired by those who view 
CBE as a major social/educational concept responsive to so many ills inher- 
ent in our existing educational system. 

CBE will not get far in the endeavor to change this sytem unless it^is 
able to move beyond what Keeton (1974) has described as a "faddish demand 
for large scale school change." No matter how strongly such change is sup>- 
ported by those who demand equity and accountability, CBE must provide 
empirical evidence that it wbrks better than the status quo if it is to 
become widely accepted. The uphill push against the existing system's 
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reticence to change (as in alJj. systems) will not be sufficiently served by 
ideological, philosophical or polemical arguments no matter how strongly 
they side with equity, accountability or other broad social goals. The- 
cuccomes must speak for themselves. 
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